RHAIENG-2846 Prefetch: COMPONENT_DIR from Makefile, local build default, Tekton discovery#3044
Conversation
github-actions
bot
added
the
review-requested
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
📝 WalkthroughWalkthroughDerives COMPONENT_DIR from the Makefile, tightens Makefile prefetch/local-build logic, adds Tekton YAML auto-discovery for NPM in prefetch scripts, makes download-npm a no-op when no npm entries, updates README to variant-aware prefetched-input paths and docs, and adjusts several lockfile/platform entries. Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes 🚥 Pre-merge checks | ✅ 3 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
scripts/lockfile-generators/prefetch-all.sh (1)
245-252: Prefer selected variant first when resolving Tekton file.Line 245 currently probes RHDS before ODH unconditionally. Using the active
VARIANTas primary lookup would reduce mismatch risk if npm paths diverge per variant later.♻️ Suggested variant-priority lookup
-if [[ -d ".tekton" ]] && command -v yq &>/dev/null; then - tekton_file=$(find_tekton_yaml "$COMPONENT_DIR" "rhds" 2>/dev/null | head -1) || true - if [[ -z "$tekton_file" ]]; then - tekton_file=$(find_tekton_yaml "$COMPONENT_DIR" "odh" 2>/dev/null | head -1) || true - fi +if [[ -d ".tekton" ]] && command -v yq &>/dev/null; then + primary_variant="$VARIANT" + fallback_variant=$([[ "$VARIANT" == "rhds" ]] && echo "odh" || echo "rhds") + tekton_file=$(find_tekton_yaml "$COMPONENT_DIR" "$primary_variant" 2>/dev/null | head -1) || true + if [[ -z "$tekton_file" ]]; then + tekton_file=$(find_tekton_yaml "$COMPONENT_DIR" "$fallback_variant" 2>/dev/null | head -1) || true + fi🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@scripts/lockfile-generators/prefetch-all.sh` around lines 245 - 252, The lookup currently always tries the "rhds" variant first; change it to prefer the active VARIANT by calling find_tekton_yaml with "$COMPONENT_DIR" and "$VARIANT" first (assigning to tekton_file), and only if that returns empty fall back to the other variant (e.g., test the alternate literal like "rhds" or "odh"); update the tekton_file resolution logic around the find_tekton_yaml calls so the active VARIANT is attempted first, then the alternate is attempted as a fallback.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@Makefile`:
- Around line 90-97: The prefetch guard currently triggers the $(error ...)
branch even for dry-run discovery (make -n) and blocks CI; update the error
conditional in the Makefile so it skips the prefetch check when a bypass flag is
set (e.g. SKIP_PREFETCH_CHECK=yes) or when MAKEFLAGS indicates a metadata-only
dry-run. Specifically, change the nested $(if ...) that emits $(error Prefetch
required for hermetic build...) to first test that SKIP_PREFETCH_CHECK is empty
(and/or that $(filter -n,$(MAKEFLAGS)) is false) before emitting the error; keep
the existing BUILD_DIR, LOCAL_BUILD_ARG and CACHI2_VOLUME logic intact. Ensure
the new condition allows calling make -n SKIP_PREFETCH_CHECK=yes <target> to
bypass the prefetch check.
---
Nitpick comments:
In `@scripts/lockfile-generators/prefetch-all.sh`:
- Around line 245-252: The lookup currently always tries the "rhds" variant
first; change it to prefer the active VARIANT by calling find_tekton_yaml with
"$COMPONENT_DIR" and "$VARIANT" first (assigning to tekton_file), and only if
that returns empty fall back to the other variant (e.g., test the alternate
literal like "rhds" or "odh"); update the tekton_file resolution logic around
the find_tekton_yaml calls so the active VARIANT is attempted first, then the
alternate is attempted as a fallback.
ℹ️ Review info
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Cache: Disabled due to data retention organization setting
Knowledge base: Disabled due to data retention organization setting
📒 Files selected for processing (7)
.github/workflows/build-notebooks-TEMPLATE.yamlMakefilecodeserver/ubi9-python-3.12/prefetch-input/odh/artifacts.lock.yamlcodeserver/ubi9-python-3.12/uv.lock.d/pylock.cpu.tomlscripts/lockfile-generators/README.mdscripts/lockfile-generators/download-npm.shscripts/lockfile-generators/prefetch-all.sh
ysok
force-pushed
the
RHAIENG-2846-post-hermetic-cleanup
branch
from
4d5f08f to
a7ee952
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (2)
scripts/lockfile-generators/prefetch-all.sh (2)
245-252: Prefer selected variant first, then fallback to the other variant.NPM Tekton discovery currently tries RHDS first unconditionally. Using
$VARIANTas first choice keeps behavior aligned with caller intent.Proposed patch
- tekton_file=$(find_tekton_yaml "$COMPONENT_DIR" "rhds" 2>/dev/null | head -1) || true + tekton_file=$(find_tekton_yaml "$COMPONENT_DIR" "$VARIANT" 2>/dev/null | head -1) || true if [[ -z "$tekton_file" ]]; then - tekton_file=$(find_tekton_yaml "$COMPONENT_DIR" "odh" 2>/dev/null | head -1) || true + fallback_variant=$([[ "$VARIANT" == "rhds" ]] && echo "odh" || echo "rhds") + tekton_file=$(find_tekton_yaml "$COMPONENT_DIR" "$fallback_variant" 2>/dev/null | head -1) || true fi🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@scripts/lockfile-generators/prefetch-all.sh` around lines 245 - 252, The Tekton YAML discovery currently hardcodes trying "rhds" first then "odh"; change it to try the caller-selected variant first by calling find_tekton_yaml with "$VARIANT" (e.g., find_tekton_yaml "$COMPONENT_DIR" "$VARIANT") and if that returns empty fall back to the other variant (compute the alternate like "rhds" vs "odh" and call find_tekton_yaml again), storing the result in tekton_file; keep using the same head -1 and || true behavior.
94-95: Use null-safe selector in yq query to explicitly handle missing structure.The current
.spec.params[]selector fails on all 29 pull-request.yaml files since they lack the.spec.paramsstructure entirely. While error suppression masks this, the query is brittle and relies on2>/dev/nullfor correctness. The proposed fix with.spec.params[]?and// emptyexplicitly handles missing structures and is more robust:Proposed patch
- dockerfile_path=$(yq -r '.spec.params[] | select(.name == "dockerfile") | .value' "$f" 2>/dev/null) + dockerfile_path=$(yq -r '.spec.params[]? | select(.name == "dockerfile") | .value // empty' "$f" 2>/dev/null || true)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@scripts/lockfile-generators/prefetch-all.sh` around lines 94 - 95, The yq query used to populate dockerfile_path is brittle when .spec.params is missing; update the yq expression in the dockerfile_path assignment (the command that sets the dockerfile_path variable) to use a null-safe selector like .spec.params[]? and append a fallback (// empty) so missing structures return empty instead of erroring, removing reliance on 2>/dev/null for correctness; keep the rest of the conditional ([[ -n "$dockerfile_path" ]] || continue) unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@scripts/lockfile-generators/prefetch-all.sh`:
- Around line 245-252: The Tekton YAML discovery currently hardcodes trying
"rhds" first then "odh"; change it to try the caller-selected variant first by
calling find_tekton_yaml with "$VARIANT" (e.g., find_tekton_yaml
"$COMPONENT_DIR" "$VARIANT") and if that returns empty fall back to the other
variant (compute the alternate like "rhds" vs "odh" and call find_tekton_yaml
again), storing the result in tekton_file; keep using the same head -1 and ||
true behavior.
- Around line 94-95: The yq query used to populate dockerfile_path is brittle
when .spec.params is missing; update the yq expression in the dockerfile_path
assignment (the command that sets the dockerfile_path variable) to use a
null-safe selector like .spec.params[]? and append a fallback (// empty) so
missing structures return empty instead of erroring, removing reliance on
2>/dev/null for correctness; keep the rest of the conditional ([[ -n
"$dockerfile_path" ]] || continue) unchanged.
ℹ️ Review info
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Cache: Disabled due to data retention organization setting
Knowledge base: Disabled due to data retention organization setting
📒 Files selected for processing (7)
.github/workflows/build-notebooks-TEMPLATE.yamlMakefilecodeserver/ubi9-python-3.12/prefetch-input/odh/artifacts.lock.yamlcodeserver/ubi9-python-3.12/uv.lock.d/pylock.cpu.tomlscripts/lockfile-generators/README.mdscripts/lockfile-generators/download-npm.shscripts/lockfile-generators/prefetch-all.sh
✅ Files skipped from review due to trivial changes (1)
- codeserver/ubi9-python-3.12/uv.lock.d/pylock.cpu.toml
🚧 Files skipped from review as they are similar to previous changes (2)
- scripts/lockfile-generators/download-npm.sh
- codeserver/ubi9-python-3.12/prefetch-input/odh/artifacts.lock.yaml
ysok
force-pushed
the
RHAIENG-2846-post-hermetic-cleanup
branch
from
a7ee952 to
bcc6c30
Compare
|
All Konflux builds are good, but failed with OOM while pushing to quay.io |
| # Derive component dir from Makefile (single source of truth). Works for all | ||
| # image targets (codeserver, jupyter-*, runtime-*, rstudio-*, base-images-*). | ||
| COMPONENT_DIR=$(make -n "${{ inputs.target }}" 2>&1 | sed -n 's/.*#\*# Image build directory: <\([^>]*\)>.*/\1/p' | head -n1) | ||
| if [ -z "$COMPONENT_DIR" ]; then | ||
| echo "Could not derive COMPONENT_DIR from Makefile for target ${{ inputs.target }}" >&2 | ||
| exit 1 | ||
| fi |
There was a problem hiding this comment.
+1, it's essentially this same thing in Python
notebooks/ci/cached-builds/gha_pr_changed_files.py
Lines 66 to 67 in 3c7287e
There was a problem hiding this comment.
The component target doesn't mapped well with component directory, they are different in other images. Once we have time, we should re-org that code base, so we can easily get component_dir from giving image target, instead of doing this regex.
| # Used where we need an empty expansion (avoids undefined variable warning). | ||
| empty := |
There was a problem hiding this comment.
+1, my ai advised me the same thing
ysok
force-pushed
the
RHAIENG-2846-post-hermetic-cleanup
branch
from
bcc6c30 to
5692b1b
Compare
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
@ysok: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@scripts/lockfile-generators/README.md`:
- Line 69: Fix the typo in the README line that reads "# OR using Local podman
build (Please scroll downn to Local podman build section for more detail.)" by
changing "downn" to "down" so the sentence reads "...Please scroll down to Local
podman build section for more detail."; update the heading/text where "Local
podman build" appears to correct the misspelling.
ℹ️ Review info
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Cache: Disabled due to data retention organization setting
Knowledge base: Disabled due to data retention organization setting
📒 Files selected for processing (8)
.github/workflows/build-notebooks-TEMPLATE.yamlMakefilecodeserver/ubi9-python-3.12/prefetch-input/odh/artifacts.lock.yamlcodeserver/ubi9-python-3.12/uv.lock.d/pylock.cpu.tomlscripts/lockfile-generators/README.mdscripts/lockfile-generators/download-npm.shscripts/lockfile-generators/prefetch-all.shtests/containers/workbenches/jupyterlab/jupyterlab_datascience_test.py
🚧 Files skipped from review as they are similar to previous changes (1)
- scripts/lockfile-generators/download-npm.sh
…lt, Tekton discovery - GHA: Derive COMPONENT_DIR from Makefile; fail clearly if missing. - Makefile: Always pass LOCAL_BUILD=true for hermetic targets; require cachi2/output when prefetch-input exists; fix empty-variable warning. - prefetch-all.sh: Add find_tekton_yaml (ODH/RHDS by dockerfile param); remove --tekton-file and package-lock fallback; allow running without variant dir. - download-npm.sh: Exit 0 when Tekton file has no npm entries. - Regenerate artifacts.lock.yaml and pylock.cpu.toml; update README.
ysok
force-pushed
the
RHAIENG-2846-post-hermetic-cleanup
branch
from
5692b1b to
30d1bed
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
scripts/lockfile-generators/README.md (1)
728-761: Good addition: Git submodule documentation.The new section provides clear, step-by-step instructions for setting up Git submodules for hermetic builds. The example using code-server is concrete and helpful.
Minor clarity suggestion: Line 754's
cd ../..could be confusing since the user hascd'd into a nested path. Consider specifying the target directory explicitly:📝 Suggested clarification
# Commit the submodule and .gitmodules -cd ../.. +cd "$(git rev-parse --show-toplevel)" # return to repo root git add -A . git commit -m "Added submodule code-server"🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@scripts/lockfile-generators/README.md` around lines 728 - 761, In the "## 5. Git submodule" section the command `cd ../..` is ambiguous after `cd codeserver/ubi9-python-3.12/prefetch-input/code-server`; replace that ambiguous step with an explicit instruction to return to the repository root (or explicitly name the target directory `codeserver/ubi9-python-3.12`/repository root) so readers know exactly where to run `git add`/`git commit`; update the example around `codeserver/ubi9-python-3.12/prefetch-input/code-server` and the `cd ../..` reference to clearly indicate the intended directory.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/containers/workbenches/jupyterlab/jupyterlab_datascience_test.py`:
- Line 87: The pytest.skip message is mistyped and has an extra trailing quote;
update the skip call in the test (the pytest.skip(...) line referencing
datascience_image.name and datascience_image.labels['name']) to a clear,
correctly worded reason such as "Image {datascience_image.name} does not have
-rstudio- in {datascience_image.labels['name']}" (remove the stray trailing
single quote and change "does have" to "does not have").
---
Nitpick comments:
In `@scripts/lockfile-generators/README.md`:
- Around line 728-761: In the "## 5. Git submodule" section the command `cd
../..` is ambiguous after `cd
codeserver/ubi9-python-3.12/prefetch-input/code-server`; replace that ambiguous
step with an explicit instruction to return to the repository root (or
explicitly name the target directory `codeserver/ubi9-python-3.12`/repository
root) so readers know exactly where to run `git add`/`git commit`; update the
example around `codeserver/ubi9-python-3.12/prefetch-input/code-server` and the
`cd ../..` reference to clearly indicate the intended directory.
ℹ️ Review info
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Cache: Disabled due to data retention organization setting
Knowledge base: Disabled due to data retention organization setting
📒 Files selected for processing (8)
.github/workflows/build-notebooks-TEMPLATE.yamlMakefilecodeserver/ubi9-python-3.12/prefetch-input/odh/artifacts.lock.yamlcodeserver/ubi9-python-3.12/uv.lock.d/pylock.cpu.tomlscripts/lockfile-generators/README.mdscripts/lockfile-generators/download-npm.shscripts/lockfile-generators/prefetch-all.shtests/containers/workbenches/jupyterlab/jupyterlab_datascience_test.py
🚧 Files skipped from review as they are similar to previous changes (3)
- codeserver/ubi9-python-3.12/prefetch-input/odh/artifacts.lock.yaml
- .github/workflows/build-notebooks-TEMPLATE.yaml
- scripts/lockfile-generators/download-npm.sh
| pytest.skip( | ||
| f"Image {datascience_image.name} does have -rstudio- in {datascience_image.labels['name']=}'" | ||
| ) | ||
| pytest.skip(f"Image {datascience_image.name} does have -rstudio- in {datascience_image.labels['name']=}'") |
There was a problem hiding this comment.
Fix skip message typo/trailing quote.
The skip reason has awkward wording and an extra trailing ', which makes test output noisy.
Suggested patch
- pytest.skip(f"Image {datascience_image.name} does have -rstudio- in {datascience_image.labels['name']=}'")
+ pytest.skip(f"Image {datascience_image.name} has '-rstudio-' in name label: {name_label}")🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/containers/workbenches/jupyterlab/jupyterlab_datascience_test.py` at
line 87, The pytest.skip message is mistyped and has an extra trailing quote;
update the skip call in the test (the pytest.skip(...) line referencing
datascience_image.name and datascience_image.labels['name']) to a clear,
correctly worded reason such as "Image {datascience_image.name} does not have
-rstudio- in {datascience_image.labels['name']}" (remove the stray trailing
single quote and change "does have" to "does not have").
3 participants
RHAIENG-2846 Post-hermetic cleanup
Summary by CodeRabbit
Documentation
Chores
Data
Tests